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(54) Image sensing device processing image signals in blocks 

(57) Control to read signals from a sensor unit hav- 
ing a plurality of pixels by pixel blocks of a predeter- 
mined size, and the read signals are processed into 
pseudo luminance and color signals by pixel blocks of 
the predetermined size. 
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Description 

-BACKGROUND OF THE INVENTION 

[0001] The present invention relates to an image 
sensing device for obtaining color image signalS: an im- 
age sensing apparatus and an image sensing system 
using the image sensing device, an image processing 
method, and a computer readable storage medium stor- 
ing program codes of the image processing method. 
[0002] Conventionally, there is an image sensing ap- 
paratus which acquires image signals (color image sig- 
nals) of an object using an image sensing device, cov- 
ered with a color filter, on which an optical Innage of the 
object is formed, applies image processes, such as 
compression and expansion, to the acquired image sig- 
nals, and displays the processed image signals and/or 
records the processed image signals In a storage me- 
dium. 

[0003] In compression processing performed in such 
an image processing apparatus, image signals output- 
ted from the image sensing device are divided into 
blocks of a predetermined size, applied with, e.g., dis- 
crete cosine transformation, quantization, and/or varia- 
ble length coding by blocks, and compressed data is ob- 
tained. 

[0004] As processes to be performed in advance of 
image processes, such as compression, (referred to as 
"pre-processing" hereinafter) signals output from the im- 
age sensing device (specifically, if color filters covering 
the image sensing device are complementary color fil- 
ters, the output signals are Ye (yellow), Cy (cyan), Mg 
(magenta) and Gr (green) raw signals, and if they are 
primary color filters, the output signals are R (red). G 
(green) and B (blue) raw signals) are temporarily stored 
in frame memory or line memory, applied with color 
processes, such as while balance correctbn and y cor- 
rection, then a luminance signal Y and color difference 
signals U and V are obtained on the basis of the proc- 
essed signals. Then, the aforesaid image processes, 
such as compression, are applied to the luminance sig- 
nal Y and the color difference signals U and V obtained 
as a result of the pre-processing. 
[0005] Further, as another example of the image proc- 
esses, besides compression, performed alter the pre- 
processing, there is a process tor determining whether 
or not a given color Is a complexion (complexion recog- 
nition) for discriminating a human in an image. 
[0006] Fig. 22 is a block diagram illustrating a brief 
configuration of a conventional image sensing appara- 
tus capable of performing the complexion recognition. 
As shown in Fig. 22, the image sensing apparatus con- 
verts analog image signals outputtcd from a CCD (im- 
age sensor) 101 into digital signals in an analog-digital 
(A/D) converter 102, and stores a predetermined 
amount of digital signals (e.g., data of two lines) in line 
mennory 1 03. 

[0007] Thereafter, the image data stored in the line 



memory 103 is applied with color processes, such as 
white balance correction and y correction, by blocks of. 
e.g., 2x2 pixels in a color processing circuit 104, and 
a luminance signal Y and color difference signals U and 

s V are generated. Then, the cobr difference signals U 
and V are provided to a complexion determination circuit 
105, where whether a color, expressed by the color dif- 
ference signals U and V, is a complexion or not is deter- 
mined on the basis of the color difference signals U and 

10 V. 

[0008] in the aforesaid conventional image sensing 
apparatus, Image signals are serially outputted; there- 
fore, the frame memoiy or line memory for the pre- 
processing is necessary. Further, time for waiting until 
^5 Image signals necessary for pre-processing are stored 
in the frame memory or line memory is required. Fur- 
thermore, considerably large'number of operations are 
to be performed in the pre-processing, which has limited 
to increase throughput. 
20 [0009] Thus, the image processes, such as compres- 
sion and complexion determination, which are subse- 
quent processes to the pre-processing are not per- 
formed efficiently in the conventbnal image sensing ap- 
paratus. 

2S 

SUMMARY OF THE INVENTION 

[0010] The present invention has been made in con- 
siderat bn of the above situation, and has as its object 

30 to provide an image sensing device capable of perform- 
ing high precision image processes efficiently. 
[0011] According to the present Invention, the image 
sensing device comprises a sensor unit having a plural- 
ity of pixels; control means for controlling to read signals 

35 from the sensor unit by pixel blocks of a predetermined 
size; and signal processing means for acquiring lumi- 
nance and color difference informatbn on tfie basis of 
the signals, read from the sensor unit, by pixel blocks of 
the predetermined size. 

40 [0012] It is another object of the present invention to 
provide an image sensing device capable of performing 
color determination processing for a given pixel. 
[0013] According to the present invention, the image 
sensing device further comprises color determination 

45 means for determining whether or not the color differ- 
ence Information acquired by the signal processing 
means represents predetermined color 
[0014] Further, it is still another object of the present 
invention to provide an image sensing apparatus and 

so an image sensing system using either of the aforesaid 
image sensing devices, an image processing method, 
and a computer readable storage medium storing pro- 
gram codes of the image processing method. 
[0015] According to the present inventbn, the image 

ss sensing apparatus includes a sensor unit having a plu- 
rality of pixels; control means for controlling to read sig- 
nals from the sensor unit by pixel blocks of a predeter- 
mined size; signal processing means for acquiring lumi- 
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nance and color difference information on Ihe basis of 
the signals, read from the sensor unit, by pixel blocks of 
the predetermined size; and compression means for 
compressing the luminance and color difference infor- 
mation calculated by the signal processing means. 
[0016] According to the present invention, the image 
processing method comprises a reading step of reading 
signals of a plurality of pixels by pixel blocks of a prede- 
termined size; and a signal processing step of acquiring 
luminance and color difference information on the basis 
of the signals read by pixel blocks of the predetermined 
size in the reading slep. 

[001 7] According to the present invention, the compu- 
ter readable storage medium includes program codes 
of the steps of either of the aforesaid image processing 
methods. 

[001 8] Other features and advantages of the present 
invention will be apparent from the following description 
taken in conjunction with the accompanying drawings, 
in which like reference characters designate the same 
or similar parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0019] The accompanying drawings, which are incor- 
porated in and constitute a part of the specification, il- 
lustrate embodin^nts of the invention and. together with 
the description, serve to explain the principles of the in- 
vention. 

Fig. 1 is a block diagram illustrating an image sens- 
ing system according to a first embodiment of the 
present invention; 

Fig. 2 is a block diagram illustrating a configuration 
of a sensor unit according to the first embodiment 
of the present invention; 

Fig. 3 is an explanatory view showing raw signals 
read by four-pixel blocks; 

Fig. 4A is an explanatory view showing raw signals 
read by eight-pixel blocks; 
Fig. 4B is an explanatory view showing signals 
stored in 2 x 2 analog memory; 
Fig. 5 is an explanatory view showing signals stored 
in the 2 X 2 analog memory when primary color fil- 
ters are used; 

Fig. 6 is a block diagram illustrating another config- 
uration of the sensor unit shown in Fig. 1 according 
to a second embodiment of the present invention; 
Fig. 7 is an explanatory view for explaining a posi- 
tion of block where data to be stored in 6 X 6 analog 
memory represents; 

Fig. 8 is an explanatory view for explaining an out- 
put order of data from the sensor unit when code 
book compression technique is used in a compres- 
sion unit; 

Fig. 9 is an explanatory view for explaining data to 
be coded in Ihe code book compression technique; 
Fig. 10 is an explanatory view for explaining reso- 
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lutfon of data obtained by shifting the position of a 
block by two pixels; 

Fig. 11 is a block diagram illustrating a configuration 
of a sensor unit according to a third embodiment of 
5 the present inventbn; 

Fig. 12A is a diagram illustrating an example of a 
basic unit of primary color filters; 
Fig. 1 2B is a diagram illustrating an example of a 
basic unit of complementary color filters; 
Fig. 13 Is a flowchart showing operation of a lumi- 
nance/color diflerence pre-processing unit and a 
complexion determination unit according to the third 
embodiment of the present inventbn; 
Fig. 14 is a graph for explaining complexion deter- 
mination performed by the complexion determina- 
tion unit according to the third embodiment of the 
present invention; 

Fig. 15 is a circuit diagram illustrating a configura- 
tion of a two-dimensional image sensor, a two-line 
image memory, and a reading circuit, shown in Fig. 
1 1 , according to the third embodiment of the present 
invention; 

Fig. 16 is a circuit diagram illustrating a configura- 
tion of a luminance/color difference pre-processing 
unit and a complexbn determination circuit, shown 
in Fig. 11, according to the third embodiment of the 
present invention; 

Fig. 17 is a circuit diagram illustrating another con- 
figuration of the two-dimensional image sensor, the 
two-line image memory, and the reading circuit, 
shown in Fig. 1 1 . according to the third embodiment 
of the present invention; 

Fig. 18 is a circuit diagram illustrating another con- 
figuration of the luminance/color difference pre- 
processing unit and the complexion determination 
circuit, shown in Fig. 11 , according to the third em- 
bodiment of the present invention; 
Figs. 1 9A to 1 9D are explanatory view for explain ing 
an example of image processing which uses the 
complexion determination result; 
Fig. 20 is a block diagram illustrating a configuration 
of an image sensing system according to a fourth 
embodiment of the present Invention; 
Fig. 21 is a block diagram illustrating a configuration 
of an Image sensing apparatus according to a fifth 
embodiment of the present invention; and 
Fig. 22 Is a btock diagram illustrating a configuration 
of a conventional image sensing apparatus whk:h 
performs complexion determination. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0020] Preferred embodiments of the preseni inven- 
tion will be described in detail betow in accordance with 
the accompanying drawings. 
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<First Embodiment> 

[0021] Fig. 1 is a block diagram illustrating an image 
processing system 100 to which the present invention 
is applied. In the image processing system 100. an im- s 
age sensing apparatus (transmission side) which com- 
prises an image sensing IC chip (referred to as "sensor 
unit" hereinafter) 1 0 and a compression unit 20 performs 
data communication with a reproduction apparatus (re- 
ceiving side) which comprises an expansion unit 30, a 
color processing unit 40, anda display/recording unit 50. 
[0622] First, an operational sequence of the Image 
sensing system 100 is explained. 
[0023] In the image sensing apparatus, the sensor 
unit 10 includes an image sensingdevice (light receiving 
device) 1 1 , covered with a color filler, on which an optical 
image of an object Is formed, anda (uminance/color dif- 
ference pre-processing unit 1 3 for generating pseudo (i. 
e., unprocessed) luminance signal Y' and color differ- 
ence signals U' and V on the basis of raw signals out- 
putted from the image sensing device 11 . 
[0024] In the first embodiment, complementary color 
filters of yellow (Ye), cyan (Cy), magenta (Mg), and 
green (G) are used; therefore, a pseudo luminance sig- 
nal Y' and pseudo color difference signals U' and V are 
generated on the basis of the raw signals of four com- 
plementary colors Ye, Cy, Mg, and Gr, and outputted 
from the sensor unit 10. 

[0025] The compression unit 20 applies compression 
processing using information compression technique, 
such as JPEG. fy/IPEG, H.261 . or vector quantization, to 
the pseudo luminance signal Y' and the pseudo color 
difference signals U' and V outputted from the sensor 
unit 10. 

[0026] In the first embodiment, the compressbn unit 
20 applies discrete cosine transform (DOT) by pixel 
blocks to the pseudo luminance signal Y' and the pseu- 
do color difference signals U' and V. thereafter, further 
quantizes the transformed signals Y', U' and V. thereby 
obtaining compressed data from which a high-frequen- 
cy component is removed. Thereafter, the compression 
unit 20 encodes the compressed data. As for the coding 
method, any coding method may be used; especially, 
variable length coding in which codes having different 
lengths are assigned to data depending upon frequency 
of the data appearance can improve compression rate. 
[0027] The compressed and encoded data by the 
compression unit 20 is transmitted to the expansion unit 
30 of the repi eduction apparatus (receiving side) via 
medium, such as a communication line. 
[0028] In the reproduction apparatus, the expansion 
unit 30 performs an inverse operation of the DOT and 
quntization which had been performed in the compres- 
sion unit 20 on the data transmitted from the image 
sensing apparatus, thereby reproducing the pseudo lu- 
minance signal Y' and the color difference signals U' and 

v. 

[0029] The color processing unit 40 performs various 



necessary corrections, such as color corrections of 
white balance correction and 7 correction, for improving 
quality of an image on the pseudo luminance signal Y' 
and the color difference signals U' and V obtained in the 
expansion unit 30. As a result, luminance signal Y and 
color difference signals U and V are generated. 
[0030] The display/recording unit 50 displays and/or 
records the luminance signal Y and the color difference 
signals U and V generated by the cobr processing unit 
40 on a display device and/or a recording medium. 
[0031] In the image sensing system 100 having the 
aforesaid configuration, the distinctive configuration ac- 
cording to the first embodiment of the present Invention 
is the sensor unit 10. 

[0032] Specifically, the sensor unit 10 has a configu- 
ration as shown in Fig. 2 and includes a two-dimensional 
image sensor 201 having a plurality of pixels (photodi- 
odes) for receiving light from an object through the com- 
plementary color fillers, a reading circuit 202 for scan- 
ning the pixels ol the image sensor 201 . converting the 
charges of the pixels into electric signals (raw signals) 
of Ye, Cy, Mg. and Gr and outputting them, 2X2 analog 
memory 203 where signals sequentially outputted from 
the reading circuit 202 are stored, and matrix circuits 
204^ to 2043 for performing predetermined matrix oper- 
ation on the signals stored in the memory 203. generat- 
ing pseudo luminance signal Y' and pseudo color differ- 
ence signals U* and V* and outputting them. The forego- 
ing elements are formed on a single IC chip. 
[0033] The sensor 201 and the reading circuit 202 cor- 
respond to the image sensing device 11 of Fig. 1. and 
the memory 203 and the matrix circuits 204^ to 2043 cor- 
respond to the luminance/color difference pre-process- 
ing unit 13 of Fig. 1. 

[0034] The sensor unit 10 as described above oper- 
ates in the following manner 

[0035] First, an optical image of an object incidents 
on the sensor 201 . At this time, the optical image passes 
through the complementary color filters (not shown) of 
Ye, Cy, Mg, and Gr. 

[0036] The reading unit 202 scans the pixels of the 
sensor 201 and converts charges accumulated in the 
respective pixels into electric signals (raw signals). Ac- 
cordingly, raw signals Ye corresponding to light passed 
through yellow color filters, raw signals Cy correspond- 
ing to light passed through cyan color fillers, raw signals 
Mg corresponding to light passed through magenta 
color filters, and raw signals Gr corresponding to light 
passed through green color filters are outputted from the 
reading unit 202. 

[0037] At this time, the reading circuit 202 scans the 
sensor 201 by four-pixel blocks of 2 x 2 pixel array. For 
instance, as shown in Fig. 2. pixels which belong to a 
block are scanned, then pixels which belong to a 
block B2, which is shifted from the block B, by a pixel in 
the horizontal directbn, are scanned. Thereafter, pixels 
which belong to a block 83. which is shifted from the 
block B2 by a pixel in the horizontal directbn. are 
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scanned, and so on. 

[0038] The reading circuit 202 obtains the raw signals 
Ye. Cy. Mg, and Gr by four-pixel blocks in the aforesaid 
manner, and the signal are stored in the 2 X 2 analog 
memory 203, as shown in Fig. 3. 
[0039] The four raw signals Ye, Cy, l^g, and Gr stored 
in the memory 203 are outputted to the matrix circuits 
204, to 2043 parallel. 

[0040] The matrix circuits 204-, to 2043 respectively 
perform matrix operation on the raw signals Ye, Cy, Mg. 
and Gr outputted from the memory 203 and acquire the 
pseudo luminance signal Y' and the pseudo color differ- 
ence signals U' and V. 

[0041 ] More specifically, the matrix circuit 204^ calcu- 
lates the pseudo luminance signal Y' using, e.g., the fol- 
lowing equation (1), 

Y'=: Ye 1 MG+Cy + Gr (1) 

The matrix circuit 2042 calculates the pseudo color dif- 
ference signal U' using, e.g., the folbwtng equation (2). 

U' = (Ye + Mg) - (Cy + Gr) (2) 

Further, the matrix circuit 2043 calculates the pseudo 
color difference signal V using, e.g.. the following equa- 
tion (3), 

V = (Cy + Mg) - (Ye + Gr) (3) 

[0042] Therefore, the pseudo luminance signal Y' and 
the pseudo color difference signals U' and V obtained 
by the matrix circuits 204^ to 2043 are outputted from 
the sensor unit 1 0, and provided to the compression unit 
20 of Fig. 1 in parallel. 

[0043] According to the first embodiment as de- 
scribed above, the lumlnance/cobr difference pre- 
processing unit 13 for acquiring the pseudo luminance 
signal Y' and the pseudo color difference signals U' and 
V* from the raw signals of Ye, Cy. Mg and Gr outputted 
from the image sensing device 1 1 is lormed on the same 
chip as the image sensing device 11. Accordingly, it is 
possible to improve operation speed for obtaining the 
pseudo luminance signal Y' and the pseudo color differ- 
ence signals U* and V*. Consequently, image processes, 
such as compression processing and expansion 
processing, are performed efficiently 
[0044] Further, time saved by increasing the opera- 
tion speed for obtaining the pseudo luminance signal Y* 
and the pseudo color difference signals U' and V may 
be used for the image processes, such as compression 
processing and expansion processing; in this case, it is 
possible to improve image quality. 
[0045] Further, color processes, such as white bal- 
ance correction and 7 correction, for improving image 



quality are performed by the color processing unit 40 
after compression and encoding by the compression 
unit 20 and expansion by the expansion unit 30 are per- 
formed on the pseudo luminance signal Y' and the pseu- 
s do color difference signals U' and V outputted from the 
sensor unit 1 0. Therefore, deterioration of an image due 
to block noise and high frequency noise caused by com- 
pression and expansion is reduced to a minimum, the 
amount of information to be transmitted via a communi- 

10 cation line is greatly reduced, and deterioration of image 
signals, processed with color processes, due to com- 
pression/expansion is restrained; accordingly, it is pos- 
sible to obtain a high quality image. 
[0046] It should be noted that the sensor 201 shown 

IS in Fig. 2 is scanned by the reading circuit 202 by four- 
pixel blocks (e.g.. B,, B2. B3) of 2 X 2 pixels and four 
raw signals Ye, Cy, Mg, and Gr are obtained at a time; 
however, the present Invention is not limited to this, and 
it is possible to scan the image sensor 201 by. e.g., 

20 eight-pixel blocks ol 2 x 4 pixels as shown in Fig. 4A to 
obtain raw signals of two signals of Ye, two signals of 
Cy, two signals of Mg, and two signals of Gr at a time. 
[0047] In this case, the reading circuit 202 performs 
predetermined operation as shown In Fig. 48 (additbn 

2S of signals of two adjoining pixels in the vertical direction, 
in this case), obtains four signals (Ye + Mg), (Cy + Gr), 
(Ye + Gr), and (CY + MG) from the eight raw signals 
read from the Image sensor 201 , and stores the lour sig- 
nals to the 2 X 2 anatog memory 203. The matrix circuits 

30 204^ to 2043 respectively acquire pseudo luminance 
signal pseudo luminance signal Y and pseudo color dif- 
ference signals U' and V using, e.g., the aforesaid equa- 
tions (1) to (3) on the basis of the four signals (Ye + Mg), 
(Cy + Gr), (Ye + Gr). and (CY + MG) stored in the mem- 

35 ory 203. 

[0048] Further, in the first embodiment, the sensor 
201 is scanned by 2 x 2 pixel blocks while shilting the 
position of a block by a pixel, however, it is also possible 
to sequentially scan the sensor 201 by 2 x 2 pixel blocks 
40 so that the adjoining blocks do not overlap each other 
(In other words, the position of a block shifts by two pix- 
els). 

[0049] Further, in the image sensor 201 shown in Fig. 
2, the complementary color filters are used; however, 
primary color filters may be used insload. 
[0050] In this case, the reading circuit 202 obtains raw 
signals R corresponding lo light passed through red 
color filters, raw signals Gr corresponding to light 
passed through green color filters, and raw signals B 
so corresponding to light passed through blue color filters, 
and the raw signals R, G and B are stored In the memory 
203 as shown in Fig. 5. Then, the matrix circuit 204, 
calculates the pseudo luminance signal Y' using, e.g.. 
the following equation (4). 

55 

Y' = R + 2G + B (4) 
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The matrix circuit 2042 calculates the pseudo color dil- 
ference signal U' using, e.g., the following equation (5). 

U' = R - Y' (5) 

Further, the matrix circuit 2043 calculates the pseudo 
color difference signal V using, e.g., the following equa- 
tion (6). 

V = B - Y' (6) 

[0051] Furthermore, in the first embodiment, the DCT 

and the variable length coding are performed in the com- 
pression unit 20 as compression processing, however, 
the present invention is not limited to these, and code 
book compression technique (vector quantization tech- 
nique), tor instance, may be used instead. 
[0052] When using the code book compression tech- 
nique, the compression unit 20 first compares the pseu- 
do luminance signal Y' and the pseudo color difference 
signals U* and V which are obtained in the sensor unit 
10 to a plurality of codes (patterns) which are stored in 
advance, searches the pattern closest to the pseudo lu- 
minance signal Y' and the pseudo color difference sig* 
nals U* and V, then transmits the code number corre- 
sponding to the searched closest pattern to the expan- 
sion unit 30 of the reproduction apparatus. 
[0053] In turn, the expansion unit 30 looks up the pat- 
tern corresponding to the code number, transmitted 
from the compression unit 20, from a plurality of codes 
which are stored in advance; thereby reproduces image 
signals compressed by the compression unit 20. 

<Second Embodimenl> 

[0054] In the second embodiment, the sensor unit 10 
shown in Fig. 1 has a configuration as shown in Fig. 6. 
Since the configuration is different from the one shown 
in Fig. 2, in order to distinguish the sensor unit 10 of the 
second embodiment from that of the first embodiment, 
the sensor unit 10 is referred to by 10' hereinafter. 
[0055] In the sensor unit 10', three analog memories 
205-, to 2063 corresponding to the three matrix circuits 
204., to 2043 are further provided in addition to the con- 
figuration shown in Fig. 2. Further, in Fig. 6. the units 
and elements as those shown in Fig. 2 are referred to 
by the same reference numerals and explanation of 
them is omitted, 

[0056] The analog memories 205^ to 2083 are includ- 
ed in the iuminance/cobr difference pre-processing unit 
1 3 shown in Fig. 1 according to the second embodiment. 
[0057] The sensor unit 10* having this configuration 
operates as follows. 

[0058] First, similarly to the sensor unit 1 0 shown In 
Fig. 2, the reading circuit 202 scans the image sensor 
201 by four-pixel blocks (B^, B2. and soon) while shifting 



the position of a block by a pixel, thereby raw signals 
Ye, Cy. Mg and Gr are obtained and stored in the 2 x 2 
analog memory 203. The matrix circuits 204^ to 2043 
operates the signals Ye. Cy. Mg and Gr stored in the 
s memory 203 using the aforesaid equations (1 ) to (3) to 
obtain a -pseudo luminance signal Y' and pseudo color 
difference signals U' and V. 

[0059] The pseudo luminance signal Y' and the pseu- 
do color difference signals U' and V obtained by the ma- 

10 irix circuits 204, to 2043 are respectively stored in the 
analog memories 205, to 2053 at the same address. 
[0060] The analog memories 205, to 2063 have the 
same configuration, and have a size suitable to the com- 
pression unit 20 provided after the sensor unit 10. In the 

?5 second embodiment, each of the analog memories 205^ 
to 2053 is 6 X 6 analog memory, as an example. 
[0061] More specifically, in the analog memory 205, 
as shown in Fig. 6, for instance, the pseudo luminance 
signal Y' generated from the raw signals of Ye, Cy. Mg 

20 and Gr corresponding to the pixels included In the first 
block B, of the image sensor 201 (i.e.. an output from 
the matrix 204,) is stored at the upper leftmost address 
D, of the anatog memory 205,. Thus, the data stored at 
the address D, conceptually represents the center of 

25 mass O, of the block B, as shown In Fig. 7. 

[0062] Similarly, a pseudo luminance signal Y' gener- 
ated from the raw signals of Ye. Cy, Mg and Gr corre- 
sponding to the pixels Included in the second block B2 
is stored at the address Dg next to the address D,. The 

30 data stored at the address represents the center of 
mass O2 of the block Bg as shown in Fig. 7. 
[0063] Thus, pseudo luminance signals Y* acquired 
by the matrix circuit 204, are sequentially stored at cor- 
responding addresses of the analog memory 205, in the 

3S same manner. 

[0064] Similarly, the color difference signals U' and V 
are also stored in the analog memories 2052 and 2053 
at corresponding 6x6 addresses. 
[0065] Then, the pseudo luminance signals Y' and 

40 color difference signals U' and V corresponding to 6 x 
6 pixels stored In the respective analog memories 205, 
to 2053 are outputted from the sensor unit 10*. 
[0066] Accordingly, the pseudo luminance signals Y' 
and color difference signals U' and V enter the compres- 

45 sion unit 20, provided after the sensor unit 10', by 6 X 
6 pixels, which Is suitable tor compression processing, 
in parallel or series. 

[0067] According to the second embodiment as de- 
scribed above, the sensor unit 10* provides the pseudo 
so luminance signals Y' and color difference signals U* and 
V to the compression unit 20 by a size suitable tor com- 
pression processing; thereby it is possible to further im- 
prove throughput. 

[0068] Specifically, If the code book compression 
55 technique Is adopted in the compression unit 20. the 
size of the analog memories 205, to 2053 is set larger 
than the size of codes (e.g. ,4X4 pixels) used by the a 
code book, and may be set to 6 X 6 pixels. 



6 



11 



EP 0 969 674 A2 



12 



[0069] Accordingly, the pseudo luminance signals Y' 
and the color difference signals U' and V are sequen- 
tially outputted by the size (e.g. .6x6 pixels) necessary 
for searching the code book for a code from the sensor 
unit 10'. 

[0070] In outputting the signals from the analog mem- 
ories 205i to 2063, signals of the central 4x4 pixels 
out of 6 X 6 pixels are outputted first, as shown in Fig. 
8. then the signals of 20 outer pixels are outputted. 
Thereafter, the compression unit 20 searches the code 
book for a code corresponding to the central 4X4 pixels 
out of the 6 X 6 pixels outputted as shown in Fig. 8 so 
that the central 4X4 pixels of a set of 6 X 6 pixels adjoin 
the central 4X4 pixels of the next set of 6 x 6 pixels 
as shown in Fig. 9. 

[0071] Accordingly, the compression unit 20 can start 
searching lor a code in the code book method using pix- 
el signals of a 6 x 6 pixel block without waiting too long 
before signals of 6 x 6 pixels are outputted by blocks, 
compared to the conventional image sensing apparatus 
which serially outputs pixel signals by line. 
[0072] It is possible to sequentially scan the image 
sensor 201 once for each pixel by 2 x 2 pixel blocks (in 
other words, the position of a block shifts by two pixels. 
More specifically, referring to Fig. 6, after scanning the 
block B^, the block B2 is skipped, and the block B3 is 
scanned); however, it is preferable to scan the image 
sensor 201 while shifting the position of the block by a 
pixel. 

[0073] This is because, if the image sensor 201 is 
scanned once for each pixel by shifting the position of - 
the block by two pixels, obtained data stored in each 6 
X 6 analog memory results in representing every other 
pixels of the image sensor 201 as shown in Fig. 10; 
which deteriorates resolutbn. In thiscase, the resolution 
is decreased to 1/4 comparing to a case of scanning the 
image sensor 201 while shifting the position of a block 
by a pixel, since. In the latter case, data stored In each 
6x6 analog memory represents every pixel of the im- 
age sensor 201, as shown in Fig. 7. 

<Third Embodiment> 

[0074] in the third embodiment, another example of 
the sensor unit Is explained. 

[0075] Fig. 11 is a block diagram illustrating a config- 
uration of a sensor unit 310 according to the third em- 
bodiment of the present invention. Referring to Fig. 11, 
the sensor unit 310 includes a two-dimensional image 
sensor (photoelectric conversion unit) 311, two-line im- 
age memory 312; a reading circuit 313; a luminance/ 
color difference pre-processing unit 314; and a com- 
plexion determination circuit 31 5, all of which are formed 
on a single IC chip. 

[0076] The sensor unit 310 of the third embodiment 
as described above has color filters covering the image 
sensor 31 1 for generating a pseudo (unprocessed) lu- 
minance signal Y' and pseudo color difference signals 



U' and \r. 

[0077] Fig. 1 2A shows an example of a basic unit of 
color fillers provided over the image sensor 311 , more 
specifically, the Bayer arrangement of primary color fil- 
s ters (R, G^, Gg. B). In this arrangement, a psoudo lumi- 
nance signal Y' is obtained by using, e.g., the following 
equation (7), 

,p Y* = 0.3R + 0.59G-i-0.11B (7) 

and a pseudo color difference signal U' is obtained by 
using, e.g.. the following equation (8), 

IS 

U* = R-Y' (8) 

further, a pseudo color difference signal V Is obtained 
by using, e.g., the following equation (9) 

20 

V' = B-r (9) 

[0078] Further. Fig. 12B shows an example of a basic 
25 unit of complementary color filters. In this arrangement, 
a pseudo luminance signal Y' and pseudo color differ- 
ence signals U' and V are obtained by using the afore- 
said equations (1 ) to (3). Namely, the pseudo luminance 
signal Y' is obtained by 

30 

Y' = Mg+Gr + Cy + Ye{=2B + 3G +2R) (1) 

Further, a pseudo color difference signal U' is obtained 
35 by 

U' = (Mg + Ye) - (Gr + Cy) {= 2R - G) (2) 

40 and a pseudo color difference signal V is obtained by 

V = (Gr + Ye) - (Mg + Cy) (= 2B - G) (3) 

45 [0079] The pixel signals outputted from the image 
sensor 31 1 which is converted by the color fillers having 
the aforesaid color arrangement are provided to the two- 
line image memory 312 where signals of two lines are 
stored. Then, the stored signals are read out by the ba- 
50 sic units of the color fillers (i.e., 2 x 2 pixels) by the read- 
ing circuit 31 3 and provided to the luminance/color dif- 
ference pre-processing unit 314. 
[0080] The luminance/color difference pre-process- 
ing unit 314 performs matrix operation using the equa- 
55 tlons (7) to (9) or (1) to (3) on the pixel signals Inputted 
by the basic blocks, generates pseudo luminance signal 
Y* and pseudo color difference signals U' and V. and 
outputs the generated signals to outside as well as out- 
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puts the pseudo color difference signals U' and V to the 
complexion determination circuit 315 as shown in step 
SI o\ the flowchart in Fig. 13. 

[0081] Next, in step S2, the complexion determination 
circuit 315 determines whether a color represented by 
the pseudo cobr difference signals U' and V* is complex- 
ion or not by inputted pixel blocks (2x2 pixel blocks). 
In the third embodiment, threshold values for determin- 
ing complexion are preset in the complexion determina- 
tion circuit 315. 

[0082] Specifically, as shown in Fig. 1 4. a first thresh- 
old A and a second threshold B are predetermined for 
the (R - Y) component, and a third threshold -D and a 
fourth threshold -C are predetermined for the (B - Y) 
component for the determination. 
[0083] If the inputted pseudo color difference signals 
U* and V and the four thresholds A, B, -C and -D satisfy 
the relationship, A < U' < B and -D < V < -C, then the 
complexion determination circuit 31 5 determines that 
the color of the pixel block is complexion. In other words, 
the complexion determination circuit 315 determines 
whether or not the inputted signals U' and V are within 
the area in Fig 14 indicated by an oblique stripe pattern. 
[0084] Fig, 1 5 is a circuit diagram illustrating a config- 
uration of the two-dimensional image sensor 311 cov- 
ered with primary color filters, the two-line image mem- 
ory 312: and the reading circuit 313. Referring to the 
circuit shown in Fig. 15, when a line becomes active, 
then charge accumulated in a pixel is transferred to 
a vertical output line V^, charge accumulated in a pixel 
G^;^ is transferred to a vertical output line Vg, charge ac- 
cumulated in a pixel R4 is transferred to a vertical output 
line V3, and charge accumulated in a pixel is trans- 
ferred to a vertical output line V4. Note, the charges (pix- 
el data) accumulated in the respective pixels are also 
referred to by the reference of corresponding pixels 
hereinafter. 

[0085] Next, when a gate line S^ of a transistor for 
reading is activated, the pixel data transferred to the re- 
spective vertical output lines to V4 are stored in con- 
densers C^, C3, C5 and C7. which functions as memory. 
More specifically, the pixel data is stored in the con- 
denser C^. the pixel data G^/^ is stored in the condenser 
C3. the pixel data R4 is stored in the condenser C5, and 
the pixel data 64^ is stored in the condenser C7. 
[0086] Thereafter, the gale line is deactivated and 
the vertical output lines to V4 are reset, then a line 
H2 is activated. In turn, charge accumulated in a pixel 
G^B is transferred to a vertical output line V^, charge ac- 
cumulated in a pixel is transferred to a vertical output 
line V2, charge accumulated in a pixel G4B is transferred 
to a vertical output line V3, and charge accumulated in 
a pixel B4 is transferred to a vertical output line V4. 
[0087] Next, when a gate line S2 of a transistor for 
reading is activated, the pixel data G^g on the vertical 
output line is stored in the condenser C2. the pixel 
data Bi on the vertical output line Vg is stored in the con- 
denser C4, the pixel data G4B on the vertical output line 



is stored in the condenser C5. and the pixel data B4 
on the vertical output line V4 is stored in the condenser 

[0088] After pixel data of two lines are stored in the 
s condensers to C3 (altogether configure the two-line 
image memory 31 2) in the aforesaid manner, reading 
operation by 2 x 2 pixel blocks is initiated. Specifically, 
when a gate line S3 for transistors of the reading circuit 

313 is activated, then the pixel data R,, G^g. 6-,^ and 
10 Bi stored in the condensers to C4 are outputted in 

parallel. 

[0089] Next, when a gate line S4 for transistors of the 
reading circuit 313 is activated, then the pixel data ^4, 
G4B, G4A and B4 stored in the condensers C5 to Cq are 
t5 outputted in parallel. The pixel data of the two-dimen- 
sional image sensor 311 is thus read out by 2 X 2 pixel 
blocks. 

[0090] Fig. 16 is a circuit diagram illustrating a config- 
uration of the luminance/color difference pre-processing 

20 unit 314 and the complexion determination circuit 315. 
Referring to Fig. 16, after pixel data R^, G^q. G^;^ and 
Bi are inputted to the luminance/cok>r difference pre- 
processing unit 31 4, the pixel data G^^andG^^ are add- 
ed then multiplied by 0.5 to obtain the average. Further, 

^5 (he average is multiplied by O.Sg, the pixel data R^ is 
multiplied by 0.3, and the pixel data 6^ is multiplied by 
0.11, and the obtained products are added, thereby a 
pseudo luminance signal Y* is generated. Thereafter, 
operations (R - Y') and (B - Y') are performed and pseu- 

30 do color difference signals U' and V are generated. 
[0091] As described above, the acquired pseudo lu- 
minance signal Y' and the pseudo cobr difference sig- 
nals U' and V are outputted to outside of the sensor unit 
310 as well as the pseudo color difference signals U' 

35 and V* are provided to the complexion determination cir- 
cuit 31 5. The threshold values A, B, -C and -D are preset 
in the complexion determination circuit 315, as already 
described above, and the pseudo color difference sig- 
nals U' and V are compared to these threshold values. 

40 and when both of the conditions of A < U' < B and -D < 
V < -C are satisfied, then the complexion determination 
circuit 31 5 determines the color of the pixel block repre- 
sented by the input pseudo color difference signals U* 
and V Is complexion. 

45 [0092] Fig. 17 is a circuit diagram illustrating another 
configuration of the two-dimensional image sensor 311 
covered with complementary cotor fillers, the two-line 
image memory 31 2, and the reading circuit 31 3, and Fig. 
18 is a circuit diagram illustrating another configuration 

50 of the luminance/color difference pre-processing unit 

31 4 and the complexion determination circuit 31 5 when 
the complementary cok)r filters are used. The circuit 
shown in Fig. 17 operates in the same manner as the 
circuit shown in Fig. 15, and the circuit shown In Fig 18 

55 operates in the same manner as the circuit shown in Fig. 
16; therefore, the explanation of the operations is omit- 
ted. 

[0093] Figs. 19A to 19D are explanatory view for ex- 
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plaining an example of image processing which uses the 
complexion detenmination result. 
[0094] Referring to Figs. 1 9A to 1 9D. in a reproduced 
Image as shown in Fig. 1 9A, the complexion determina- 
tion circuit 315 outputs flags for a complex ion area, 
hatched In Fig. 19B. Then, an area Including the com- 
plexion area is set in accordance with predetermined 
conditions as shown in Fig. 19C, and the face can be 
easily and precisely clipped from the reproduced image 
as shown in Fig. 19D. 

[0095] Note, in the third embodiment, an example of 
determining complexion is explained; however, it is pos- 
sible to determine arbitrary color other than complexion 
by setting different threshold values in the complex bn 
determination circuit 315. 

[0098] Further, in the third embodiment, the complex- 
ton determination Is performed by basic units of the color 
filters, however, it is possible to obtain an average of 
pixel data of an n x m pixel block anddetermine whether 
or not the color of the pixel block is complexion from the 
pseudo color difference signals U' and V which are gen- 
erated on the basis of the average. 
[0097] According to the third embodiment as de- 
scribed above, color is determined by pixel blocks of a 
predetermined size, thus, detection of arbitrary color in 
image signals outputted from the image sensor is per- 
formed easily and precisely. 

[0098] Further, according to the third embodiment, 
color pre-processing, i.e., calculation of luminance and 
color difference signals, performed in general in ad- 
vance of color image processes is performed in the sen- 
sor unit, it is possible to improve throughput as well as 
reduce the required memory size for performing the 
color image processes which are performed after the 
pre-processing. 

[0099] Note, the image sensor described in the third 
embodiment is also applicable to the first and second 
embodiments, and vice versa. 

<Fourth Embodlment> 

[0100] Next, the lourth embodiment to which the sen- 
sor unit 310 of the third embodiment is applied Is ex- 
plained with reference to a block diagram shown in Fig. 
20. 

[0101] Fig. 20 is a block diagram illustrating a config- 
uration of an image sensing system using the sensor 
unit 310 of the third embodiment. Referring to Fig. 20, 
an image sensing apparatus (transmission side) of the 
image sensing system comprises the sensor unit 310. 
a code -book-method compression unit 340, and a code 
number output unit 341, and a reproduction apparatus 
(receiving side) of the image sensing system comprises 
a code number input unit 350, a code-book-method ex- 
pansion unit 360, a color processing unit 400 and a dis- 
play/recording unit 390. 

[01 02] The sensor unit 310 comprises an image sens- 
ing device 411 including the two-dimensional image 



sensor 311. the iwo-line image menrKjry 312 and the 
reading circuit 313 which are shown in Fig. 11. and an 
image signal pre-processing unit 421 Including the lu- 
minance/color difference pre-processing unit 314 and 

s the complexion determination circuit 315 which are 
shown in Fig. 11. The sensor unit 310 outputs image 
signals IM (pseudo luminance signal Y' and pseudo 
color difference signals U' and V), and complexion de- 
tection signals (flag). 

10 [0103] The image signals IM outputted from the sen- 
sor unit 310 enter the code-book-method compression 
unit 340. The compression unit 340 compares the image 
signals IM of a predetermined number of pixels inputted 
from the sensor unit 310 to a plurality of codes (patterns) 

'5 stored in code lx>ok memory (not shown). If the image 
signals IMof 1000 X 1 000 pixels are outputted, the com- 
pression unit 340 compares the innage signals IM by 4 
X 4 pixel blocks, for instance, to code patterns of the 
code book if the code pattern is of a 4 x 4 array. 

20 [01 04] In the code book memory of Ihc fourth embod- 
iment, a plurality of codes having a predetermined size 
are stored, and, for searching for a code closest to the 
image signals, the compression unit 340 pre-searches 
codes representing complexion. The codes represent- 

25 ing complexion may be stored in connection with a pre- 
determined sign, or may be selected after determining 
colors expressed by the codes. Among the p re- 
searched codes, the code having the highest correlation 
with the inputted Image signals is selected, and the code 

30 number of the selected code is outputted. The code 
number outputted from the compression unit 340 is 
transmitted from the code number output unit 341 to the 
code number input unit 350 via a medium, such as a 
connmunication .line. 

35 [0105] The code number transmitted via, e.g., the 
communication line enters the code number input unit 
350 and provided to the code -book- method expansion 
unit 360. The code -book-method expansion unit 360 
reads a pattem (code) corresponding to the input code 

40 number from code book memory (not shown), thereby 
reproduces image data compressed by the code-book- 
method compression unit 340. 
[0106] The image signals IM reproduced by the ex- 
pansion unit 360 is then provided to the color processing 

45 unit 400. The color processing unit 400 has a circuit 382 
for performing various necessary corrections, such as 
cotor corrections of white balance correctbn and 7 cor- 
rectran, for improving quality of an image on cobr infor- 
mation of the image signals IM. 

50 [01 07] Thus, the raw signals Ye, Cy, Mg and Gr input- 
ted from the expansion unit 360 are applied with the pre- 
detemnined color processes by the color processing unit 
400; as a result, a luminance signal Y and color differ- 
ence signals U and V are generated and outputted. 

55 [0108] The luminance signal Y and the cotor differ- 
ence signals U and V outputted from the color process- 
ing unit 400 are provided to the display/recording unit 
390 where displayed on a display device and/or record- 
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ed on a recording medium. 

[0109] Note, in the lourth embodiment, the code book 
method is used as information compression/expansion 
method, however, the present invention is not limited to 
this, and compression/expansion by discrete cosine s 
transformation, quantization, variable length coding, for 
instance, may be performed instead. 
[0110] According to the image sensing system of the 
fourth embodiment as described above, color correction 
processes for obtaining a high quality image are per- 
formed after the expansion process not before compres- 
sion process which is a conventional sequence. There- 
fore, in addition to the advantages of the third embodi- 
ment, deterioration of an image due to block noise and 
high frequency noise caused by compression and ex- 
pansion is reduced to a minimum, the amount o1 infor- 
mation to be transmitted via a communication line is 
greatly reduced, and deterioration of image signals, 
processed with color processes, due to compression/ 
expansion is restrained; accordingly, it is possible to ob- 
tain a high quality image. 

[0111] Further, since image signals are oulputted by 
pixel blocks of a predetermined size suitable to com- 
pression processing from a sensor unit, it is possible to 
greatly increase precision and processing speed for 
compressing image signals outputted from the sensor 
unit. 

[0112] Note, the image sensing system described in 
the fourth embodiment is also applicable to the first and 
second embodiments, and vice versa. 

<Fifth Embodiment> 

[0113] Next, the fifth embodiment will be explained 
with reference Fig. 21 . In the fourth embodiment, the 
compressed signals are outputted externally via a me- 
dium, such as a communication line. In contrast, in the 
fifth embodiment, the compressed signals are tempo- 
rarily recorded on a recording medium, reproduced, and 
applied with coior processes after being expanded. 
[0114] The difference between Figs. 20 and 21 is that 
a write unit 430, a storage medium 431, and a read unit 
432 are provided between the code-book-method conr>- 
pression unit 340 and the code-book-method expansion 
unit 360. 

[0115] By configuring the apparatus as described 
above, required capacity of internal memory is further 
reduced since image signals IM from the image sensing 
device 310 are stored in the storage medium 431. In ad- 
dition, since color processes, such as white balance cor- 
rection and Y correction, are performed by the color 
processing unit 400 after the image signals, read from 
the storage medium 431 , are expanded by the code- 
book-method expansion unit 360. deterioration of an im- 
age renoains low, and it is possible to display a high qual- 
ity image on a display device 440. 
[0116] It should be noted that the image sensing ap- 
paratus according to the fifth embodiment may not in- 



clude the read unit 432 and its subsequent units. In this 
case, the read unit 432. the code-book-method expan- 
sion unit 360. the color processing unit 400. and the dis- 
play/recording unit 390 may be included In a reproduc- 
ing apparatus (e.g.. a personal computer). 
[0117] Further the code book method is used as com- 
pression/expansbn method in the fourth embodiment, 
however, the present invention is not limited to this, and 
corrpressbn/expansion by discrete cosine transforma- 
tion, quantization, variable length coding, for instance, 
may be performed instead. 

[0118] According to the fifth embodiment as described 
above, it is possible to achieve the similar advantages 
as those of the fourth embodiment. 

<Other Embodiment> 

[0119] The object of the present invention can also be 
achieved by providing a storage medium storing pro- 
gram codes for performing the aforesaid processes to 
a computer system or apparatus (e.g., a personal com- 
puter), reading the program codes, by a CPU or MPU 
of the computer system or apparatus, from the storage 
medium, then executing the program. 
[0120] In this case, the program codes read from the 
storage medium realize the functions according to the 
embodiments, and the storage medium storing the pro- 
gram codes constitutes the invention. 
[0121] Further, the storage medium, such as a floppy 
disk, a hard disk, an optical disk, a nDagneto-optical disk. 
CD-ROM, CD-R. a magnetic tape^ a non-volatile type 
memory card, and ROM can be used lor providing the 
program codes. 

[0122] Furthermore, besides the aforesaid functions 
according to the above embodiments are realized by ex- 
ecuting the program codes which are read by a compu- 
ter, the present invention includes a case where an OS 
(operating system) or the like working on the computer 
performs a part or the entire processes in accordance 
with designations of the program codes and realize 
functions according to the above embodiments. 
[0123] Furthennore, the present inventbn also in- 
cludes a case where, after the program codes read from 
the storage medium are written in a function expansion 
card which is inserted Into the computer or in a menoory 
provided in a f unctk^n expansion unit which is connected 
to the computer, CPU or the tike contained in the func- 
tion expansion card or unit performs a part or the entire 
process in accordance with designations of the program 
codes and realizes functions of the above embodi- 
ments. 

[0124] The present invention is not limited to the 
above embodiments and various changes and modifi- 
cations can be made within the spirit and scope of the 
present invention. Therefore to appraise the publk; of 
the scope of the present Invention, the following claims 
are made. 
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Claims 

1 . An image sensing device characterized by compris- 
ing: 

a sensor unit (201 , 311 ) having a plurality ot pix- 
els; 

control means (202, 31 2, 31 3) tor controlling to 
read signals from said sensor unit by pixel 
blocks of a predetermined size; and 
signal processing means (13, 204, - 2043. 314) 
for acquiring luminance and color difference in- 
formation on the basis of the signals, read from 
said sensor unit, by pixel blocks of the prede- 
termined size. 

2. The image sensing device according to claim 1, 
wherein said control means (202) controls to read 
signals by pixel blocks so that at least two adjoining 
pixel blocks in the horizontal or vertical direction 
partially overlap each other. 

3. The image sensing device according to claim 1 or 
.2, further comprising color determination means 

(31 5) for determining whether or not the color dif- 
ference information acquired by said signal 
processing means (314) represents predetermined 
color. 

4. The image sensing device according to claim 3, 

wherein a determination result by said color deter- 
mination means (315) is outputted in synchroniza- 
tion with a time of outputting the luminance and 
color difference information from said signal 
processing means (314). 

5. The image sensing device according to claim 3 or 
4, wherein first to fourth threshold values, TH1 to 
TH4, are preset in said color determination means 
(315), and, for a (R-Y) component and a (B-Y) com- 
ponent of the color difference information^ when 
both equations 

TH1 < (R-Y) < TH2 

TH3 < (B-Y) < TH4 

are satisfied, said color detenmination means deter- 
mines that the color difference information repre- 
sents the predetermined color. 

6. The image sensing device according to any of 
claims 3 to 5, wherein the predetermined color is 
complexion. 

7. The image sensing device according to any of 



claims 1 to 6. further comprising memory (205^ - 
2063) for storing the luminance and color difference 
informatk)n acquired by said signal processing 
means (13, 204^ - 2043) representing a predeter- 
s mined amount for a pixel group. 

8. The image sensing device according to claim 7, 
wherein the luminance and color difference infor- 
mation acquired by said signal processing means 

10 (13, 204, - 2043) is outputted to an external signal 
processing apparatus, and the predetermined 
amount of the information to be stored in said mem- 
ory Is set depending upon the external signal 
processing apparatus. 

75 

9. The image sensing device according to claim 1 , 
wherein said sensor unit (201,311), said control 
means (202, 312, 313) and said signal processing 
means 1 3, 204, - 2043, 31 4) are formed on a single 

20 ic chip. 

10. The image sensing device according to claim 3, 
wherein said sensor unit (311): said control means 
(312, 313), said signal processing means (314), 

25 and said color determination means (315) are 
formed on a single IC chip. 

11. The image sensing device according to claim 7, 
wherein said sensor unit (201), said control means 

30 (202), said signal processing means (13. 204^ - 
2043), and said memory (205i - 2053) are formed 
on a single IC chip. 

1 2. An image sensing apparatus characterized by com- 
35 prising: 

a sensor unit (201 , 31 1 ) having a plurality of pix- 
els; 

control means (202, 312, 313) for controlling to 
40 read signals from said sensor unit by pixel 

blocks of a predetermined size; 
signal processing means (1 3, 204, - 2043, 31 4) 
for acquiring luminance and color difference in- 
formation on the basis of the signals, read from 
45 said sensor unit, by pixel blocks of the prede- 

termined size; and 

compression means (20. 340) for compressing 
the luminance and color difference information 
calculated by said signal processing means. 

so 

13. The image sensing apparatus according to claim 
1 2. wherein said control means (202) controls to 
read signals by pixel blocks so that at least two ad- 
joining pixel blocks in the horizontal or venical di- 

55 rection partially overlap each other. 

14. The image sensing apparatus according to claim 1 2 
or 1 3, further comprising color determination means 



11 



21 



EP 0 969 674 A2 



22 



(315) for determining whether the color difference 
information acquired by said signal processing 
means (314) represents predetermined color. 

1 5. The image sensing apparatus according to claim 4, 
wherein a determination result by said color deter- 
mination means (315) is outputted in synchroniza- 
tion with a time of outputting the luminance and 
color difference information from said signal 
processing means (314). 

16. The image sensing apparatus according to claim 1 4 
or 15, wherein first to fourth threshold values. TH1 
to TH4, are preset In said color determination 
means (315), and, fofB (R-Y) componentanda (B- 
Y) component of the color difference information, 
when both equations 

TH1 <(R-Y)<TH2 



TH3 < (B-Y) < TH4 

are satisfied, said color determination means deter- 
mines that the color difference information repre- 
sents the predetermined color. 

17. The image sensing apparatus according to any of 
claims 14 to 16, wherein the predetermined cobr is 
complexion. 

18. The image sensing apparatus according to any of 
claims 12 to 17, further comprising memory (205^ - 
2053) for storing the luminance and color difference 
information acquired by said signal processing 
means (13, 204^ - 2043) representing a predeter- 
mined amount for a pixel group. 

19. The image sensing apparatus according to claim 
18, wherein the predetermined amount of the infor- 
mation to be stored in said memory (205^ - 2053) is 
set depending upon compression process per- 
formed by said compression means (20). 

20. The image sensing apparatus according to claim 
12, wherein said sensor unit (201, 311), said control 
means (202, 312, 313) and said signal processing 
means (1 3, 204-, - 2043. 31 4) are formed on a single 
IC chip. 

21. The image sensing apparatus according to claim 
14, wherein said sensor unit (311), said control 
means (312, 313), said signal processing means 
(314), and said color determination means (315) 
are formed on a single IC chip. 

22. The image sensing apparatus according to claim 



18. wherein said sensor unU (201). saidcontrol- 
means (202), said signal processing means (13, 
204, - 2043), and said memory (205, - 2053) are 
formed on a single IC chip. 

5 

23. The image sensing apparatus according to any of 
claims 12 to 22, wherein said compression means 
(20, 340) performs code-book vector quantization. 

10 24. The image sensing apparatus according to any of 
claims 12 to 22, wherein said compression means 
(20, 340) performs either discrete cosine transfor- 
mation, quantization, or variable length coding. 

25. The image sensing apparatus according to any of 
claims 12 to 24, further comprising output means 
(341 ) for outputting the luminance and color differ- 
ence information compressed by said compression 
means (340). 

20 

26. The image sensing apparatus according to any of 
claims 12 to 25, further comprising write means 
(430) for writing the luminance and color difference 
information compressed by said compression 

25 means (340) on a storage medium. 

27. The image sensing apparatus according to claim 

26, further comprising: 

30 read means (432) for reading the compressed 

luminance and color difference information 
from the storage medium; 
expansion means (360) for expanding the com- 
pressed luminance and color difference infor- 
ms matlon read by said read means; and 

color signal processing means (382) for apply- 
ing color processes including at least white bal- 
ance correction or y correction to signals ex- 
panded by said expansion means. 

40 

28. The image sensing apparatus according to claim 

27, wherein said compression means (340) per- 
forms code-book vector quantization, and said ex- 
pansion means (360) performs inverse operation of 

45 code-book vector quantization. 

29. The image sensing apparatus according to claim 
27. wherein said compression means (340) per- 
forms either discrete cosine transformation, quanti- 

50 zation. or variable length coding and said expansion 
means (360) performs inverse operation of the 
compression performed by said compression 
means. 

55 30. An image signal reading and processing method 
characterized by comprising; 

a reading step of reading signals of a plurality 
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24 



of pixels by pixel blocks of a predetermined 

size; and 

a signal processing step of acquiring luminance 
and color difference information on the basis of 
the signals read by pixel blocks of the predeter- 
mined size in said reading step. 

31 . The method according to claim 30, wherein, in said 
reading step, signals are read by pixel blocks so that 
at (east two adjoining pixel blocks in the horizontal 
or vertical direction partially overlap each other. 

32. The method according to claim 30 or 31 , further 
comprising a color determination step of determin- 
ing whether the color difference information ac- 
quired by said signal processing means represents 
predetermined color 

33. The method according to claim 32, wherein a deter- 
mination result of said color determination step Is 
outputted in synchronization with a time of output- 
ting the luminance and color difference information 
in said signal processing step. 

34. The method according to claim 32 or 33, wherein 
first to fourth threshold values, TH 1 to TH4, are pre- 
set, and, for a (R-Y) component and a (B-Y) com- 
ponent of the color difference information, when 
both equations 

TH1 < (R-Y) < TH2 

TH3 < (B-Y) < TH4 

are satisfied, it is determined in said color determi- 
nation step that the color difference information rep- 
resents the predetermined color. 

35. The method according to any of claims 32 to 34, 
wherein the predetermined color is complexion. 

36. The method according to any of claims 30 to 35, 
further comprising a storing step of storing the lu- 
minance and color difference information acquired 
in said signal processing step representing a pre- 
determined amount for a pixel group. 

37. The method according to claim 36, further compris- 
ing an output step of outputting the luminance and 
color difference information acquired in sa\6 signal 
processing step to an external signal processing 
apparatus, wherein the predetermined amount of 
the information to be stored in said storing step is 
set depending upon the external signal processing 
apparatus. 



38. The method according to any of claims 20 to 37, 
further comprising: 

a compression step of compressing the lumi- 
5 nance and color difference information ac- 

quired in said signal processing step; 
an expansion step of expanding the luminance 
and color difference information compressed in 
said compressbn step; and 
^0 a color signal processing step of applying color 

processes including at least white balance cor- 
rection or V correction to signals expanded in 
said expansion step. 

'5 39. The method according to any of claims 30 to 37. 
lurther comprising: 

a compression step of compressing the lumi- 
nance and color difference information ac- 

^0 quired in said signal processing step, 

a write step of writing the luminance and color 
difference information compressed in said 
compression step on a storage medium; 
a read step of reading the compressed lumi- 

2S nance and cotor difference information from the 

storage medium; 

an expansion step of expanding the luminance 
and color difference information read in said 
read step; and 

30 a color signal processing step of applying color 

processes including at least white balance cor- 
rection or V correction to signals expanded in 
said expansion step. 

3S 40. The method according to claim 38 or 39, further 

comprising a storing step of storing the luminance 
and color difference information acquired in said 
signal processing step representing a predeter- 
mined amount for a pixel group, wherein the prede- 
40 termined amount of the information to be stored in 
said storing step is set depending upon compres- 
sion process performed in said compression step. 

41. The method according to claim 38 or 39, wherein, 
45 in said compression step, code-book vector quan- 
tization is performed and in said expansbn step, in- 
verse operation of code-book vector quantization is 
performed. 

so 42. The method according to claim 38 or 39. wherein, 
in said compression step, either discrete cosine 
transformation, quantization, or variable length cod- 
ing is perlormed and in said expansion step, inverse 
operation of the compression perfornned in said 
55 compression step is performed. 

43. Storage media storing instructions for controlling 
the processor to carry out the method of any of 
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claims 30 to 42. 

44. The computer program product according to claim 
43. further comprising: 

5 

second computer readable code means for 
compressing the acquired luminance and color 
difference information; 

third computer readable code means for ex- 
panding the compressed luminance and color io 
difference information; and 
fourth computer readable code means for ap- 
plying color processes including at least white 
balance correction or y correction to expanded 
signals. 

45. The computer program product according to claim 
43, further comprising: 

second computer readable code means for 
compressing the acquired luminance and color 
difference information; 

third computer readable code means for writing 
the compressed luminance and color differ- 
ence information on a storage medium; 25 
fourth computer readable code means for read- 
ing the compressed luminance and color differ- 
ence infomnation from the storage medium; 
fifth computer readable code means for ex- 
panding the compressed luminance artd color 30 
difference information; and 
sixth computer readable code means for apply- 
ing color processes including at least while bal- 
ance correction orycorrection to expanded sig- 
nals. 35 
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